There are a number of places where people have already broken python 2 compatibility (although I think all would be fixable if there were a reason to care); I mostly just wanted to make sure I wouldn't create an unnecessary limitation in this added code. The portion of this code needed to support it is a fairly small fraction of the whole, and aside from removing some #ifdef blocks, removing it wouldn't provide much simplification.

There is one test failure on python 3.12, however, it appears to be unrelated:

I've fixed this on the master branch. Newer versions of astropy expect the transform_to argument to be an initialized instance of the coordinate class.

Is there a particular reason we don't name the libraries using the standard convention? Seems like an easy solution would be to name the libraries the way that python expects...

That requires breaking with the conventions expected by everything else for library naming, which make it annoying to link anything else with them. The reason this situation is weirder/harder for us than for most anyone else is that combining python bindings into the same library which is used when python is not (directly) involved is highly atypical.

Can we use symlinks for one or the other?

Ping here, it sounds like people are starting to run into this issue more often with updated software. Is this PR ok to merge?

Sorry about the delay, I was trying to do a bit more checking of the other approach with symlinks to present the same libraries with two different names. That seems to fundamentally work, but there's some detail that I can't quite get right. A few notes in case we ever want to re-visit it:

The main change is to install a symlink with an ugly name (e.g. _core.so for core) in the python module directory, instead of the actual library:

--- a/cmake/Spt3gIncludes.cmake.in
+++ b/cmake/Spt3gIncludes.cmake.in
@@ -61,7 +61,9 @@ macro(add_spt3g_library lib_name)
        list(APPEND SPT3G_LIBRARIES ${lib_name})
        set(SPT3G_LIBRARIES ${SPT3G_LIBRARIES} PARENT_SCOPE)
        install(TARGETS ${lib_name} EXPORT ${PROJECT_NAME}Config LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}")
-       install(TARGETS ${lib_name} DESTINATION ${PYTHON_MODULE_DIR}/spt3g)
+       INSTALL(CODE "execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink \
+                     ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/${CMAKE_SHARED_LIBRARY_PREFIX}${lib_name}${CMAKE_SHARED_LIBRARY_SUFFIX} \
+                     ${PYTHON_MODULE_DIR}/spt3g/_${lib_name}.so)")
 endmacro(add_spt3g_library lib_name)

The leading underscore (or some equivalent mangling) is important for an import statement to be able to distinguish the compiled module from the python portion. Doing this requires changing each compiled module's name to match, i.e. tacking on a leading underscore. Each python __init__ script must be then changed to import the compiled module with its mangled name:

--- a/core/python/__init__.py
+++ b/core/python/__init__.py
@@ -1,5 +1,4 @@
-from spt3g.core.load_pybindings import load_pybindings
-load_pybindings(__name__, __path__)
+from spt3g._core import *

In testing, this works, except for importing the modules with compiled components besides core if the working driectory is the CMake build directory. My best guess is that this is because of either witchcraft or dynamic linker search paths, and I have not been able to find a way to fix it.

At any rate, it seems like we should be able to re-evaluate or change how we do this in future if we want, so I think it's safe to merge the custom loader version for now.

Sure sounds like witchcraft to me! Yeah, I think it would be nicer in the long run if we didn't have to roll our own loader, so let's revisit at some point. What about if you used a relative import instead, e.g. from .._core import *, so that the global search path doesn't come into play?

Unfortunately, the relative import yields the same error. It seems like it is finding the symlink (one would hope for a different error if not), but it's adamant that the symbol isn't in the library, even though nm says it is.

Unfortunately, the relative import yields the same error. It seems like it is finding the symlink (one would hope for a different error if not), but it's adamant that the symbol isn't in the library, even though nm says it is.

Gremlins. Ok, some more head-scratching for later, this solution is fine for the time being. Thanks for putting in the work!